Sample array coding for low-delay

ABSTRACT

The entropy coding of a current part of a predetermined entropy slice is based on, not only, the respective probability estimations of the predetermined entropy slice as adapted using the previously coded part of the predetermined entropy slice, but also probability estimations as used in the entropy coding of a spatially neighboring, in entropy slice order preceding entropy slice at a neighboring part thereof. Thereby, the probability estimations used in entropy coding are adapted to the actual symbol statistics more closely, thereby lowering the coding efficiency decrease normally caused by lower-delay concepts. Temporal interrelationships are exploited additionally or alternatively.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/639,312 filed Jun. 30, 2017, which is a continuation of U.S.application Ser. No. 15/414,356 filed Jan. 24, 2017 (now U.S. Pat. No.9,729,891 issued Aug. 8, 2017), which is a continuation of U.S.application Ser. No. 14/141,374 filed Dec. 26, 2013 (now U.S. Pat. No.9,596,469 issued Mar. 14, 2017), which is a continuation ofInternational Application No. PCT/EP2012/063929, filed Jul. 16, 2012,which claims priority from U.S. Provisional Application No. 61/508,477,filed Jul. 15, 2011, all of which are incorporated herein by referencein their entirety.

BACKGROUND OF THE INVENTION

The present application is concerned with sample array coding such aspicture or video coding.

Parallelization of encoder and decoder is very important due to theincreased processing requirements by the HEVC standard as well as by theexpected increase of video resolution. Multi-core architectures arebecoming available in a wide range of modern electronic devices.Consequently, efficient methods to enable the use of multiple-corearchitectures are necessitated.

Encoding or decoding of LCUs occurs in raster scan, by which the CABACprobability is adapted to the specificities of each image. Spatialdependencies exist between adjacent LCUs. Each LCU (largest coding unit)depends on its left, above, above-left and above-right neighbor LCUs,because of different components, for instance, motion-vector,prediction, intra-prediction and others. Due to enable parallelizationin decoding, these dependencies typically need to be interrupted or areinterrupted in state-of-the-art applications.

Some concepts of parallelization, namely wavefront processing have beenproposed. The motivator for the further study is to perform techniques,which lower the coding efficiency loss and thus reduce the burden on thebitstream for parallelization approaches in encoder and decoder.Furthermore, low-delay processing was not possible with the availabletechniques.

Thus, it is the object of the present invention to provide a codingconcept for sample arrays allowing lower delay at comparatively lesspenalties in coding efficiency.

SUMMARY

According to an embodiment, a decoder for reconstructing a sample arrayfrom an entropy encoded data stream may be configured to: entropy decodea plurality of entropy slices within the entropy encoder data stream soas to reconstruct different portions of the sample array associated withthe entropy slices, respectively, with performing, for each entropyslice, the entropy decoding along a respective entropy coding path usingrespective probability estimations, adapting the respective probabilityestimations along the respective entropy coding path using a previouslydecoded part of the respective entropy slice, starting the entropydecoding the plurality of entropy slices sequentially using an entropyslice order, and performing, in entropy decoding a predetermined entropyslice, entropy decoding a current part of the predetermined entropyslice based on the respective probability estimations of thepredetermined entropy slice as adapted using the previously decoded partof the predetermined entropy slice, and probability estimations as usedin the entropy decoding of a spatially neighboring, in entropy sliceorder preceding entropy slice at a neighboring part of the spatiallyneighboring entropy slice.

According to another embodiment, a decoder configured to reconstruct asequence of sample arrays from an entropy encoded data stream may beconfigured to: entropy decode a current frame of the entropy encodeddata stream so as to reconstruct a current sample array of the sequenceof sample arrays, perform the entropy decoding along an entropy codingpath and using probability estimations and adapt the probabilityestimations along the entropy coding path using a previously decodedpart of the current frame, wherein the decoder is configured toinitialize or determine the probability estimations for the currentframe based on probability estimations used in decoding a previouslydecoded frame of the entropy encoded data stream.

According to another embodiment, an encoder for encoding a sample arrayinto an entropy encoded data stream may be configured to: entropy encodea plurality of entropy slices into the entropy encoder data stream eachentropy slice being associated with a different portion of the samplearray, respectively, with performing, for each entropy slice, theentropy encoding along a respective entropy coding path using respectiveprobability estimations, adapting the respective probability estimationsalong the respective entropy coding path using a previously decoded partof the respective entropy slice, starting the entropy encoding theplurality of entropy slices sequentially using an entropy slice order,and performing, in entropy encoding a predetermined entropy slice,entropy encoding a current part of the predetermined entropy slice basedon the respective probability estimations of the predetermined entropyslice as adapted using the previously encoded part of the predeterminedentropy slice, and probability estimations as used in the entropyencoding of a spatially neighboring, in entropy slice order precedingentropy slice at a neighboring part of the spatially neighboring entropyslice.

According to still another embodiment, an encoder configured to encode asequence of sample arrays into an entropy encoded data stream may beconfigured to: entropy encode a current frame of the entropy encodeddata stream so as to reconstruct a current sample array of the sequenceof sample arrays, perform the entropy encoding along an entropy codingpath and using probability estimations and adapt the probabilityestimations along the entropy coding path using a previously encodedpart of the current frame, wherein the encoder is configured toinitialize or determine the probability estimations for the currentframe based on probability estimations used in encoding a previouslyencoded frame of the entropy encoded data stream.

According to another embodiment, a method for reconstructing a samplearray from an entropy encoded data stream may have the steps of: entropydecoding a plurality of entropy slices within the entropy encoder datastream so as to reconstruct different portions of the sample arrayassociated with the entropy slices, respectively, with performing, foreach entropy slice, the entropy decoding along a respective entropycoding path using respective probability estimations, adapting therespective probability estimations along the respective entropy codingpath using a previously decoded part of the respective entropy slice,starting the entropy decoding the plurality of entropy slicessequentially using an entropy slice order, and performing, in entropydecoding a predetermined entropy slice, entropy decoding a current partof the predetermined entropy slice based on the respective probabilityestimations of the predetermined entropy slice as adapted using thepreviously decoded part of the predetermined entropy slice, andprobability estimations as used in the entropy decoding of a spatiallyneighboring, in entropy slice order preceding entropy slice at aneighboring part of the spatially neighboring entropy slice.

According to another embodiment, a method configured to reconstruct asequence of sample arrays from an entropy encoded data stream may havethe steps of: entropy decoding a current frame of the entropy encodeddata stream so as to reconstruct a current sample array of the sequenceof sample arrays, performing the entropy decoding along an entropycoding path and using probability estimations and adapting theprobability estimations along the entropy coding path using a previouslydecoded part of the current frame, and wherein the method comprisesinitialize or determine the probability estimations for the currentframe based on probability estimations used in decoding a previouslydecoded frame of the entropy encoded data stream.

According to still another embodiment, a method for encoding a samplearray into an entropy encoded data stream may have the steps of: entropyencoding a plurality of entropy slices into the entropy encoder datastream each entropy slice being associated with a different portion ofthe sample array, respectively, with performing, for each entropy slice,the entropy encoding along a respective entropy coding path usingrespective probability estimations, adapting the respective probabilityestimations along the respective entropy coding path using a previouslydecoded part of the respective entropy slice, starting the entropyencoding the plurality of entropy slices sequentially using an entropyslice order, and performing, in entropy encoding a predetermined entropyslice, entropy encoding a current part of the predetermined entropyslice based on the respective probability estimations of thepredetermined entropy slice as adapted using the previously encoded partof the predetermined entropy slice, and probability estimations as usedin the entropy encoding of a spatially neighboring, in entropy sliceorder preceding entropy slice at a neighboring part of the spatiallyneighboring entropy slice.

According to another embodiment, a method for encoding a sequence ofsample arrays into an entropy encoded data stream may have the steps of:entropy encoding a current frame of the entropy encoded data stream soas to reconstruct a current sample array of the sequence of samplearrays, performing the entropy encoding along an entropy coding path andusing probability estimations and adapting the probability estimationsalong the entropy coding path using a previously encoded part of thecurrent frame, wherein the method has initializing or determining theprobability estimations for the current frame based on probabilityestimations used in encoding a previously encoded frame of the entropyencoded data stream.

Another embodiment may have a computer program having a program codeconfigured to perform, when running on a computer, the methods asmentioned above.

If the entropy coding of a current part of a predetermined entropy sliceis based on, not only, the respective probability estimations of thepredetermined entropy slice as adapted using the previously coded partof the predetermined entropy slice, but also probability estimations asused in the entropy coding of a spatially neighboring, in entropy sliceorder preceding entropy slice at a neighboring part thereof, theprobability estimations used in entropy coding are adapted to the actualsymbol statistics more closely, thereby lowering the coding efficiencydecrease normally caused by lower-delay concepts. Temporalinterrelationships may be exploited additionally or alternatively.

For example, the dependency on probability estimations as used in theentropy coding of a spatially neighboring, in entropy slice orderpreceding entropy slice may involve the initialization of theprobability estimations at the beginning of entropy coding thepredetermined entropy slice. Usually, probability estimations areinitialized to values adapted to symbol statistics of a representativeblend of sample array material. In order to avoid the transmission ofthe initialization values of the probability estimations, they are knownto encoder and decoder by convention. However, such pre-definedinitialization values are, naturally, merely a compromise between sideinformation bitrate on the one hand and coding efficiency on the otherhand as such initialization values naturally—more or less—deviate fromthe actual sample statistics of the currently coded sample arraymaterial. The probability adaptation during the course of coding anentropy slice adapts the probability estimations to the actual symbolstatistics. This process is accelerated by initializing the probabilityestimations at the beginning of entropy coding the current/predeterminedentropy slice using already adapted probability estimations of thejust-mentioned spatially neighboring, in entropy slice order precedingentropy slice as the latter values have already been, to some extent,adapted to the actual symbol statistics of the sample array currently athand. Low-delay coding may nevertheless be enabled by using, ininitializing the probability estimations for the predetermined/currententropy slices, the probability estimation used at the neighboring partthereof, rather than manifesting themselves at the end of entropy codingthe preceding entropy slice. By this measure, wavefront processing isstill possible.

Further, the above-mentioned dependency on the probability estimationsas used in the entropy coding of the spatially neighboring, in entropyslice order preceding entropy slice may involve the adaptation processof adapting the probability estimations used in entropy coding thecurrent/predetermined entropy slice itself. Probability estimationadaptation involves the use of the just-coded part, i.e., the just-codedsymbol(s), in order to adapt the current state of the probabilityestimations to the actual symbol statistics. By this measure,initialized probability estimations are adapted at some adaptation rateto the actual symbol statistic. This adaptation rate is increased byperforming the just-mentioned probability estimation adaptation not onlybased on the currently coded symbol of the current/predetermined entropyslice, but also dependent on probability estimations as manifestingthemselves at a neighboring part of the spatial neighboring, in entropyslice order preceding entropy slice. Again, by choosing the spatiallyneighborhood of the current part of the current entropy slice and theneighboring part of the preceding entropy slice appropriately, wavefrontprocessing is still possible. The benefit from coupling the ownprobability estimation adaption along the current entropy slice, withthe probability adaptation of the preceding entropy slice is theincreased rate at which the adaptation to the actual symbol statisticstakes place as the number of symbols traversed in the current andprevious entropy slices contributes to the adaptation, rather thanmerely the symbols of the current entropy slice.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will be described subsequentlyreferring to the appended drawings, in which:

FIG. 1 shows a block diagram of an exemplary encoder;

FIG. 2 shows a schematic diagram of a partitioning of a picture intoslices and slice parts (i.e. blocks or coding units) along with codingorders defined thereamong;

FIG. 3 shows a flow diagram of functionality of an exemplary encodersuch as the one of FIG. 1;

FIG. 4 shows a schematic diagram for explaining the functionality of anexemplary encoder such as the one of FIG. 1;

FIG. 5 shows a schematic diagram for a parallel operationalimplementation of encoder and decoder;

FIG. 6 shows a block diagram of an exemplary decoder;

FIG. 7 shows a flow diagram of functionality of an exemplary decodersuch as the one of FIG. 6;

FIG. 8 shows a schematic diagram for an exemplary bitstream resultingfrom the coding scheme of FIGS. 1 to 6;

FIG. 9 schematically shows an example for how to compute probabilitywith the help of others LCUs;

FIG. 10 shows a graph illustrating the RD results for Intra (4 Threads),in a comparison with HM3.0;

FIG. 11 shows a graph illustrating the RD results for Low Delay (1Thread), in a comparison with HM3.0;

FIG. 12 shows a graph illustrating the RD results for Random Access (1Thread), in a comparison with HM3.0;

FIG. 13 shows a graph illustrating the RD results for LowDelay (4Threads), in a comparison with HM3.0;

FIG. 14 schematically and exemplarily illustrates the possible compoundsof entropy slices;

FIG. 15 schematically and exemplarily illustrates a possible signalingof an entropy slice;

FIG. 16 schematically and exemplarily illustrates Encoding,segmentation, interleaving and decoding of entropy slice data viachunks;

FIG. 17 schematically and exemplarily illustrates a possible compoundbetween Frames;

FIG. 18 schematically and exemplarily illustrates a possible usage ofcollocated information;

FIG. 19 schematically shows the possibility of a wavefront runningobliquely within the spatio/temporal space spanned by consecutive samplearrays; and

FIG. 20 schematically shows another example for subdividing entropyslices into chunks.

DETAILED DESCRIPTION OF THE INVENTION

In order to ease the understanding of the below-outlined measures forimproving the achievement of low-delay at less penalties in terms ofcoding efficiency, the encoder of FIG. 1 is firstly described in moregeneral terms without, preliminarily, discussing the advantageousconcepts of embodiments of the present application and how same may bebuilt into the embodiment of FIG. 1. It should be mentioned, however,that the structure shown in FIG. 1 merely serves as an illustrativeenvironment in which embodiments of the present application may be used.Generalizations and alternatives for encoders and decoders in accordancewith embodiments of the present invention are also briefly discussed.

FIG. 1 shows an encoder for encoding a sample array 10 into an entropyencoded data stream 20. As shown in FIG. 1, the sample array 10 may beone of a sequence 30 of sample arrays and the encoder may be configuredto encode the sequence 30 into the data stream 20.

The encoder of FIG. 1 is generally indicated by reference sign 40 andcomprises a precoder 42 followed by an entropy encoding stage 44, theoutput of which outputs data stream 20. The precoder 42 is configured toreceive and operate on sample array 10 in order to describe the contentthereof by way of syntax elements of a predetermined syntax with eachsyntax element being of a respective one of a predetermined set ofsyntax element types which, in turn, are associated with a respectivesemantics.

In describing the sample array 10 using the syntax elements, theprecoder 42 may subdivide the sample array 10 into coding units 50. Theterm “coding unit” may, for reasons set out in more detail below,alternatively be called “coding tree units” (CTU)s. One possibility ofhow precoder 42 may subdivide the sample array 10 into coding units 50is exemplarily shown in FIG. 2. In accordance with this example, thesubdivision regularly subdivides the sample array 10 into coding units50, so that the latter are arranged in rows and columns so as togaplessly cover the complete sample array 10 without overlap. In otherwords, precoder 42 may be configured to describe each coding unit 50 byway of syntax elements. Some of these syntax elements may formsubdivision information for further subdividing the respective codingunit 50. For example, by way of multi-tree subdivisioning, thesubdivision information may describe a subdivision of the respectivecoding unit 50 into prediction blocks 52 with the precoder 42associating a prediction mode with associated prediction parameters foreach of these prediction blocks 52. This prediction subdivision mayallow for the prediction blocks 52 being different in size asillustrated in FIG. 2. The precoder 42 may also associate residualsubdivision information with the prediction blocks 52 so as to furthersubdivide the prediction blocks 52 into residual blocks 54 so as todescribe the prediction residual per prediction block 52. Thus, precodermay be configured to generate a syntax description of sample array 10 inaccordance with a hybrid coding scheme. However, as already noted above,the just-mentioned way that the precoder 42 describes the sample array10 by way of syntax elements has merely been presented for illustrationpurposes and may also be implemented differently.

The precoder 42 may exploit spatial interrelationships between contentof neighboring coding units 50 of sample array 10. For example, precoder42 may predict syntax elements for a certain coding unit 50 from syntaxelements determined for previously coded coding units 50 which arespatially adjacent to the currently coded coding unit 50. In FIGS. 1 and2, for example, the top and left-hand neighbors serve for prediction asillustrated by arrows 60 and 62. Moreover, precoder 42 may, in anintra-prediction mode, extrapolate already coded content of neighboringcoding units 50 into the current coding unit 50 so as to obtain aprediction of the samples of the current coding unit 50. As shown inFIG. 1, precoder 42 may, beyond exploiting spatial interrelationships,temporarily predict samples and/or syntax elements for a current codingunit 50 from previously coded sample arrays as illustratively shown inFIG. 1 by way of arrow 64. That is, motion-compensated prediction may beused be precoder 42, and the motion vectors themselves may be subject totemporal prediction from motion vectors of previously coded samplearrays.

That is, precoder 42 may describe the content of sample array 10 codingunit wise and may, to this end, use spatial prediction. The spatialprediction is restricted for each coding unit 50 to spatiallyneighboring coding units of the same sample array 10 such that whenfollowing a coding order 66 among the coding units 50 of sample array10, the neighboring coding units serving as prediction reference for thespatial prediction, have generally been traversed by the coding orders66 prior to the current coding unit 50. As illustrated in FIG. 2, thecoding order 66 defined among the coding units 50 may, for example, be araster scan order according to which coding units 50 are traversed rowby row from top to bottom. Optionally, a subdivision of array 10 into anarray of tiles may cause the scan order 66 to traverse—in a raster scanorder—the coding units 50 composing one tile first before preceding tothe next in a tile order which, in turn, may also be of a raster scantype. For example, the spatial prediction may involve merely neighboringcoding units 50 within a coding unit row above the coding unit rowwithin which the current coding unit 50 resides, and a coding unitwithin the same coding unit row, but to the left relative to the currentcoding unit. As will be explained in more detail below, this restrictiononto the spatial interrelationship/spatial prediction, enables parallelwavefront processing.

Precoder 42 forwards the syntax elements to the entropy coding stage 44.As just outlined, some of these syntax elements have predictively beencoded, i.e. represent prediction residuals. Precoder 42 may, thus, beregarded as a predictive coder. Beyond that, precoder 42 may be atransform coder configured to transform code residuals of the predictionof the content of the coding units 50.

An exemplary internal structure of the entropy coding stage 44 is alsoshown in FIG. 1. As shown, the entropy coding stage 44 may comprise,optionally, a symbolizer for converting each of syntax elements receivedfrom precoder 42, the number of possible states of which exceed thesymbol alphabet's cardinality into a sequence of symbols s_(i) of thesymbol alphabet based on which the entropy coding engine 44 operates.Beyond that optional symbolizer 70, the entropy coding engine 44 maycomprise a context selector 72 and an initializer 74, a probabilityestimation manager 76, a probability estimation adaptor 78 and anentropy coding core 80. The entropy coding core's output forms theoutput of entropy coding stage 44. Besides, the entropy coding core 80comprises two inputs, namely one for receiving the symbols s_(i) of thesequence of symbols, and another one for receiving a probabilityestimation p_(i) for each of the symbols.

Owing to the properties of entropy coding, the coding efficiency interms of compression rate, increases with an improvement of theprobability estimation: the better the probability estimation matchesthe actual symbol statistics, the better the compression rate.

In the example of FIG. 1, the context selector 72 is configured toselect, for each symbol s_(i), a corresponding context c_(i) among a setof available contexts managed by manager 76. It should be noted,however, that the context selection forms merely an optional feature andmay be left off with, for example, using the same context for eachsymbol. However, if using the context selection, context selector 72 maybe configured to perform the context selection at least partially basedon information concerning coding units outside the current coding unit,namely concerning neighboring coding units within the restrictedneighborhood discussed above.

Manager 76 comprises a storage which stores, for each available contextan associated probability estimation. For example, the symbol alphabetmay be a binary alphabet so that merely one probability value may haveto be stored for each available context.

The initializer 74 may, intermittently, initialize or re-initialize theprobability estimations stored within manager 76 for the availablecontexts. Possible time instants at which such initialization may beperformed, are discussed further below.

The adapter 78 has access to the pairs of symbols s_(i) andcorresponding probability estimations p_(i) and adapts the probabilityestimations within manager 76 accordingly. That is, each time aprobability estimation is applied by entropy coding core 80 so as toentropy code the respective symbol s_(i), into data stream 20, adapter78 may vary this probability estimation in accordance with the value ofthis current symbol s_(i), so that this probability estimation p_(i) isbetter adapted to the actual symbol statistics when encoding the nextsymbol which is associated with that probability estimation (by way ofits context). That is, adaptor 78 receives the probability estimationfor the selected context from manager 76 along with the correspondingsymbol s_(i) and adapts the c_(i), probability estimation p_(i)accordingly so that, for the next symbol s_(i) of the same context theadaptive probability estimation is used.

The entropy coding core 80 is, for example, configured to operate inaccordance with an arithmetic coding scheme or in aprobability-interval-partitioning-entropy coding scheme. In arithmeticcoding, entropy coding core 80 would, for example, continuously updateits state when coding the sequence of symbols, with the state beingdefined by a probability interval defined by a probability intervalwidth value and probability interval offset value, for example. Whenoperating in the pipe concept, the entropy coding core 80 would, forexample, subdivide the domain of possible values of the probabilityestimations into different intervals with performing fixed-probabilityentropy coding with respect to each of these intervals, therebyobtaining a substream for each of the sub-intervals the codingefficiency of which is respectively tailored to the associatedprobability interval. In case of entropy coding, the data stream 20output would be an arithmetically coded data stream signaling to thedecoding side information enabling emulating or redoing the intervalsubdivision process.

Naturally, it would be possible for entropy coding state 44 to entropycode all information, i.e. all syntax elements/symbols s_(i), relatingto sample array 10 with initializing the probability estimations merelyonce at the beginning thereof, and then continuously updating theprobability estimations by adaptor 78. This would, however, result in adata stream 20 which would have to be sequentially decoded at thedecoding side. In other words, there would not be a possibility for anydecoder to subdivide resulting data stream into several sub-portions anddecoding the sub-portions in parallel. This would, in turn, hamper anylow-delay efforts.

Accordingly, as will be outlined in more detail below, it is favorableto subdivide the amount of data describing the sample array 10 intoso-called entropy slices. Each of these entropy slices would,accordingly, cover a different set of the syntax elements relating tothe sample array 10. If the entropy coding stage 44 would, however,entropy encode each entropy slice completely independent of each otherby firstly initializing the probability estimation once with thencontinuously updating the probability estimation for each entropy sliceindividually, then the coding efficiency would be decreased due to theincreased percentage of the data relating to, and describing, the samplearray 10 for which the probability estimations used are (yet) lessaccurately adapted to the actual symbol statistics.

In order to overcome the just-mentioned problems in accommodating thewish for low coding delay on the one hand and high coding efficiency onthe other hand, the following encoding scheme may be used, which is nowdescribed with respect to FIG. 3.

Firstly, the data describing the sample array 10 is subdivided intoportions called “entropy slices” in the following. The subdivision 80needs not to be overlap-free. On the other hand, this subdivision may atleast partially correspond to a spatial subdivision of the sample array10 into different portions. That is, according to the subdivision 80,syntax elements describing sample array 10 may be distributed ontodifferent entropy slices depending on the location of the coding unit 50which the respective syntax element relates to. See, for example, FIG.2. FIG. 2 shows an exemplary subdivision of sample array 10 intodifferent portions 12. Each portion corresponds to a respective entropyslice. As exemplarily shown, each portion 12 corresponds to a row ofcoding units 50. Other subdivisions are, however, feasible as well.However, it is advantageous if the subdivision of the sample array 10into portions 12 follows the aforementioned coding order 66 so thatportions 12 cover consecutive runs of coding units 12 along the codingorder 66. Even if so, however, the start and end positions of portion 12along coding order 66 need not coincide with the left-hand andright-hand edges of the rows of coding units 50, respectively. Even acoincidence with the borders of coding units 50 immediately follow eachother and coding order 66 does not need to be mandatory.

By subdividing sample array 10 in that way, an entropy slice order 16 isdefined among portions 12 along which portions 12 follow each otheralong the coding order 66. Moreover, for each entropy slice, arespective entropy coding path 14 is defined, namely the fragment ofcoding path 66 running to the respective portion 12. In the example ofFIG. 2 where portions 12 coincide with the rows of coding units 50, theentropy coding paths 14 of each entropy slice point along the rowdirection, in parallel to each other, i.e. here from the left-hand sideto the right-hand side.

It should be noted that it would be possible to restrict spatialpredictions performed by precoder 42 and context derivations performedby context selector 72 so as to not cross slice boundaries, i.e. so thatthe spatial predictions and context selections do not depend on datacorresponding to another entropy slice. This way, the “entropy slices”would correspond the usual definition of “slices” in H.264, for example,which are completely decodable independently from each other, except forthe below-outlined probability initialization/adaptation dependency.However, it would also be feasible to allow spatial predictions andcontext selections, i.e. generally speaking dependencies, to cross sliceboundaries in order to exploit local/spatial inter-dependencies as WPPprocessing is still feasible even as far as the reversal of theprecoding, i.e. the reconstruction based on the syntax elements, and theentropy context selection is concerned. In so far, the entropy sliceswould somehow correspond to “dependent slices”.

Subdivision 80 may, for example, be performed by entropy coding stage44. The subdivision may be fixed or may vary among the array of sequence30. The subdivision may be fixed per convention or may be signaledwithin data stream 20.

Based on the entropy slices, the actual entropy coding may take place,i.e. 82. For each entropy slice, the entropy coding may be structuredinto a starting phase 84 and a continuation phase 86. The staring phase84 involves, for example, the initialization of the probabilityestimations as well as the triggering of the actual entropy codingprocess for the respective entropy slice. The actual entropy coding isthen performed during the continuation phase 86. The entropy codingduring phase 86 is performed along the respective entropy coding path14. The starting phase 84 for each entropy slice is controlled such thatthe entropy coding of the plurality of entropy slices startssequentially using the entropy slice order 16.

Now, in order to avoid the above-outlined penalty which would resultfrom entropy coding each entropy slice completely independent of eachother, the entropy coding process 82 is controlled such that a currentpart, e.g. a current coding unit of a current entropy slice is entropycoded based on the respective probability estimations of the currententropy slice as adapted using the previously encoded part of thecurrent entropy slice, i.e. the part of the current entropy slice to theleft of the current coding unit 50 in case of FIG. 2, and theprobability estimations as used in entropy coding the spatiallyneighboring, in entropy slice order 16 preceding entropy slice at aneighboring part, i.e. a neighboring coding unit, thereof.

In order to describe the aforementioned dependency more clearly,reference is made to FIG. 4. FIG. 4 illustrates, the n−1, n and n+1 thentropy slices in entropy slice order 16 with reference sign 90. Eachentropy slice 90 encompasses the sequence of syntax elements describingthe portion 12 of the sample array 10 which the respective entropy slice90 is associated with. Along the entropy coding path 14, entropy slice19 is segmented into a sequence of segments 92, each of whichcorresponds to a respective one of the coding units 50 of the portion 12which entropy slice 90 relates to.

As described above, the probability estimations used in entropy codingthe entropy slices 90 are continuously updated during the continuationphase 86 along the entropy coding path 14 so that the probabilityestimations increasingly better adapt the actual symbol statistics ofthe respective entropy slice 90—i.e. the probability estimations areassociated with the respective entropy slice. While the probabilityestimations 94 used to entropy code the entropy slice 90 during thecontinuation phase 86, are continuously updated, in FIG. 4, merelystates of the probability estimations 94 occurring at the start and endpositions of segments 92 are illustrated and mentioned further below. Inparticular, the state prior to entropy coding the first segment 92 asinitialized during the starting phase 84, is shown at 96, the statemanifesting itself after coding the first segment is illustrated at 98,and the state manifesting itself after encoding the first two segmentsis indicated at 100. The same elements are shown in FIG. 4 also for then−1 entropy slice in entropy slice order 16, and the following entropyslice, i.e. entropy slice n+1.

Now, in order to achieve the above-outlined dependency, the initialstate 96 for entropy coding the n-th entropy slice 90 is set dependenton any intermediate state of the probability estimations 94 manifestingitself during encoding the preceding entropy slice n−1. “Intermediatestate” shall denote any state of the probability estimations 94,excluding the initial state 96 and the final state manifesting itselfafter entropy coding the complete entropy slice n−1. By doing so,entropy coding the sequence of entropy slices 90 along the entropy sliceorder 16 may be parallelized with a degree of parallelization beingdetermined by the ratio of the number of segments 92 preceding the stateused for initialization of the probability estimations 94 for entropycoding the next entropy slice, i.e. a, and a number of segments 92succeeding this stage, i.e. b. In particular, in FIG. 4, a isexemplarily set to be equal to, with the initialization, i.e. adaptionof state 100 so as to set state 96 of current entropy slice to beequaled to state 100 of the preceding entropy slice, being illustratedby arrow 104.

By this measure, the entropy coding of any segment 92 following state100 in entropy coding path order 14 would depend on the probabilityestimation 94 as adapted during the continuation phase 86 based on thepreceding segments of the same entropy slice as well as the probabilityestimation as used in entropy coding the third segment 92 of thepreceding entropy slice 90.

Accordingly, the entropy coding of the entropy slices 90 could beperformed in parallel in a pipelined scheduling. The only restrictionsimposed onto the time scheduling would be that the entropy coding ofsome entropy slice may begin merely after finishing the entropy codingof the a-th segment 92 of the preceding entropy slice. Entropy slices 90immediately following each other in entropy slice order 16, are notsubject to any other restrictions regarding time-alignment of theentropy coding procedure during the continuation phase 86.

In accordance with another embodiment, however, a stronger coupling isused, additionally and/or alternatively. In particular, as illustratedin FIG. 4 by representative arrows 106, the probability estimationadaptation during continuation phase 86 causes that the data of thecoding unit corresponding to a certain segment 92 changes theprobability estimations 94 from the state at the beginning of therespective segment 92 till the end of this segment 92, thereby improvingthe approximation of the actual symbol statistics as denoted above. Thatis, the adaptation 106 is performed for the entropy slice n−1 merelydependent on data of entropy slice n−1, and the same applies to theprobability estimation adaptation 106 of entropy slice n, etc. Forexample, it would be possible to perform the initialization as explainedabove with respect to arrows 104 with performing the probabilityestimation adaptation 106 without any further interference between theentropy slices 90. However, in order to expedite the probabilityestimation approximation of the actual symbol statistics, probabilityestimation adaptation 106 of consecutive entropy slices may be coupledso that the probability estimation adaptation 106 of a preceding entropyslice n−1 also influences, or is taken into account, when adapting theprobability estimation adaptation of a current entropy slice n. This isillustrated in FIG. 4 by an arrow 108 pointing from state 110 of thespatially neighboring probability estimations 94 for entropy coding then−1-th entropy slice 90 to state 100 of the probability estimations 94for entropy coding the n-the entropy slice 90. When using theabove-outlined initialization of state 96, the probability adaptationcoupling 108 may be, for example, used at any of the b probabilityestimation states manifesting themselves after entropy coding the bsegments 92 of the preceding entropy slice. To be more precise, theprobability estimations manifesting themselves after entropy coding thefirst segment 92 of the current entropy slice may be the result of theusual probability adaptation 106 and by taking into account 108 of theprobability estimation states resulting from the probability estimationadaptation 106 during entropy coding the (a+1)-th segment 92 of thepreceding entropy slice n−1. The “taking into account” may, for example,involve some averaging operation. An example will be further outlinedbelow. In other words, state 98 of the probability estimations 94 forentropy coding the n-th entropy slice 90 at the beginning of entropycoding segment 92 thereof, may be the result of averaging thepredecessor state 96 of the probability estimations 94 for entropycoding the current entropy slice n as adapted using adaptation 106, andthe state before entropy coding the (a+1)-th segment 92 of the precedingentropy slice n−1 modified according to the probability adaptation 106.Analogously, state 100 may be the result of an averaging the result ofthe adaptation 106 performed during entropy coding the current entropyslice n and the result of the probability adaptation during entropycoding the (a+2)-th segment 92 of the preceding entropy slice n−1, etc.

More specifically, let

-   p(n→_({i,j}), with i,j denoting the position of any coding unit    (with (0,0) being the left, top and (I,J) the right, bottom    position), iϵ{1 . . . I} and j.epsilon.{1 . . . J}, I being the    number or columns, J being the number of rows and p( ) defining the    path order 66,-   P_({i,j}) the probability estimation used in entropy coding coding    unit {i,j}; and-   T(P_({i,j})) the result of the probability adaptation 106 of based    on coding unit {i,j};-   Then, the probability estimations 106 of consecutive entropy slices    90 may be combined to replace the usual, entropy slice internal    adaptation according to P_(p(n+1))=T(P_(p(n))), by    P _(p(n+1))=average(T(P _(p(n))),T(P _({i,j}) ₁ ), . . . ,T(P    _({i,j}) _(N) ))    where N may be 1 or more than 1 and {i,j}_(1 . . . N) is/are    selected from (lie within) any previous (in entropy slice order 16)    entropy slice 90 and its associated portion 12, respectively. The    function “average” may be one of a weighted sum, a median function,    etc. p(n)={i,j} is the current coding unit and p(n+1) follows in    accordance with the coding order 14 and 66, respectively. In the    presented embodiments p(n+1)={i+1,j}.

Advantageously, {i,j}_(1 . . . N) fulfill, for each k∈{1 . . . N},{i,j}_(1 . . . N)={i_(k,j).j_(k)} and i_(k)<i+3 and j_(k)<j withp(n)={i,j} being the current coding unit (i.e. any of the second orfollowing coding units of the current entropy slice), i.e. they do notlie beyond the wavefront.

In the latter alternative, the entropy coding time scheduling whenentropy coding the entropy slices 90 in parallel, is more closelycoupled to each other. That is, during the continuation phase 86, thenext segment 92 in line of a current entropy slice, may merely becommenced upon finishing the corresponding segment of the precedingentropy slice which is a ranking position further in entropy coding pathorder 14.

In other words, the above discussion showed an example where the decoder40, and in particular, the entropy coding stage 44, is configured toperform, for an entropy slice 90 such as the n-th, an initialization ofits probability estimations 94 before decoding the first segment 92corresponding to the first coding unit/first block 50 of the portion 12corresponding to the n-th entropy slice along the respective encodingpath 14 with probability estimations manifesting themselves after havingentropy decoded the second coding unit/block 50 of the portion 12corresponding to the, in entropy slice order 16, preceding entropy slicealong the respective encoding path 14. Additionally, or alternatively,the decoder 40 and, in particular, entropy coding stage 44, may beconfigured to perform, for each entropy slice 90, the entropy decodingand probability estimation adaptation such that, after a currentpart/block/coding unit 50 of the current entropy slice has been entropydecoded based on the respective probability estimations 94 of thecurrent entropy slice 90, the respective probability estimations 94 ofthe current entropy slice are adapted depending on this current part ofthe current entropy slice and the probability estimations as manifestingthemselves in entropy decoding a neighboring part/block/coding unit 50of the spatially neighboring, preceding entropy slice, such as the onein the above neighboring row in the second column to the right of thecurrent part/block/coding unit of the current entropy slice.

As became clear from the above discussion, probability estimation has tobe adapted/managed for each entropy slice 50 separately. This may bedone by sequentially processing the entropy slices and storingprobability estimation states as those exemplarily shown and mentionedwith respect to FIG. 4, i.e. 96, 98, 100, 110 and 102, in a respectiveprobability estimation storage 120 (see FIG. 1). Alternatively, morethan one entropy coding stage 44 may be provided so as to decode theentropy slices in parallel. This is illustrated in FIG. 5, where aplurality of instantiations of the entropy coding stage 44 are shown,each associated with a respective one of the entropy slices andcorresponding portions 12 of the sample array 10, respectively. FIG. 5also illustrates the decoding process and its possible implementationsby using parallel instantiations of respective entropy decoding stages130. Each of these entropy decoding stages 130 is fed with a respectiveone of the entropy slices conveyed via the data stream 20.

FIG. 5 shows that the entropy coding stages 44 and the respectivedecoding stages 130 do not completely operate independent of each otherin parallel. Rather, stored probability estimation states such as theones stored in storage 120, are passed from one stage corresponding to arespective entropy slice, to another stage relating to an entropy slicefollowing in accordance with the entropy slice order 16.

FIG. 5 shows for illustration purposes, also a possible traversing orderfor traversing possible subdivisions of the coding units 50 such as atraversal order 140 defined among the prediction blocks 52 within onecoding unit 50. For all these prediction blocks 52 corresponding syntaxelements are contained within the respective segment 92 andcorresponding to the values of these syntax elements, the probabilityestimations 94 are adapted during the traversal of path 140 with theadaptation during traversing coding unit 50 defining the above-mentioned“T”. In CABAC according to H.264 and HEVC, “T” is performed table-based,i.e. by a “table walk” defining transitions from a current state of theprobability estimation for a current context to a next state dependingon the current symbol value associated with that context.

Before proceeding with FIG. 6 showing a decoder embodiment correspondingto the encoder embodiment of FIG. 1, it should be noted that thepredictive nature of precoder 42 merely served as an illustrativeembodiment. In accordance with even alternative embodiments, theprecoder 42 may be left off with the syntax elements which the entropycoding stage 44 operates on, being, for example, the original samplevalues of the sample array 10. Even alternatively, the precoder 42 maybe configured to form a subband decomposition of sample array 10 such asin JPEG. The optional nature of context selector 72 has already beenmentioned above. The same applies with respect to the initializer 74.The same may be instrumented differently.

FIG. 6 shows a decoder 200 corresponding to the encoder 40 of FIG. 1. Itis visible from FIG. 6 that the construction of decoder 200substantially mirrors the construction of the encoder 40. That is,decoder 200 comprises an input 202 for receiving data stream 20,followed by a concatenation of entropy decoding stage 204 and aconstructor 206. The entropy decoding stage 204 entropy decodes theentropy slices conveyed to within data stream 202 and forwards, in turn,the decoded symbols s_(i) and syntax elements, respectively, toconstructor 206, which, in turn, requests the syntax elements fromentropy decoding stage 204 by respective request 208. In other words,constructor 206 also assumes responsibility for parsing the syntaxelement stream produced by precoder 42 within the encoder. Accordingly,constructor 206 requests sequentially the syntax elements from entropydecoding stage 204. Entropy decoding stage 204 is structuredsubstantially the same way as entropy encoding stage 44 is. Accordingly,the same reference signs of the internal blocks of entropy decodingstage 204 are re-used. The symbolizer 70, if present, turns the requestsfor syntax elements into requests for symbols and the entropy decodingcore 80 answers with a respective symbol value s_(i), symbolizer 70 mapssequences of received symbols forming valid symbol words into syntaxelements and forwards same to constructor 206. Constructor 206reconstructs the sample array 10 from the syntax element stream receivedfrom entropy decoding stage 204 such as, as outlined above, usingpredictive decoding, etc. To be more precise, constructor 206 also usesthe coding order 66 and performs the decoding coding-unit-wise withperforming predictions 60, 62 and 64. The one or more predictions forsyntax elements or for sample values are combined such as added-up,optionally with using a prediction residual obtained from the syntaxelements of the syntax element stream. The entropy decoding core 80 may,just as the entropy coding core 80, follow an arithmetic decodingconcept or a probability interval partitioning entropy decoding concept.In the case of arithmetic decoding, entropy decoding core 80 maycontinuously update an internal state in form of a partial intervalwidth value and a value such as an offset value, pointing into thispartial interval. The update is done using the inbound data stream. Thecurrent partial interval is subdivided analogously to the entropy codingcore 80 using the probability estimation p_(i) provided for each symbols_(i) by way of context selector 72 along with probability estimationmanager 76. The adapter 78 performs the probability estimationadaptation using the decoded symbol values s_(i) in order to update theprobability estimation values p_(i) of the context c_(i) associated withthe symbol s_(i) by context selector 72. The initializations byinitializer 74 are done at the same instances and in the same way as inthe encoding side.

The decoder of FIG. 6 operates in a manner very similar to thefunctionality of the encoder as described above with respect to FIG. 3.In a step 230, the sample array 10 is subdivided into entropy slices.See, for example, FIG. 8. FIG. 8 shows the data stream 20 arriving atinput 202 and in particular, entropy slices 90, contained therein. Instep 230, each of these entropy slices 90 is associated with a portion12 which the respective entropy slice 90 is associated with so that theabove-described probability estimation initialization and probabilityestimation adaptation based on the respective proceeding entropy slicemay be performed. The sample array subdivision, or, more preciselystated, the association of the entropy slices 90 with their respectiveportions 12, may be performed by constructor 206. The association may beachieved by different measures, such as side information containedwithin data stream 20 in portions not entropy coded, or by convention.

The entropy slices 90 are then, in the entropy decoding process 232,entropy decoded in a manner, mirroring the encoding process 82, namelywith performing for each entropy slice 90, a starting phase 234 and acontinuation phase 236 with probability estimation initialization andadaptation in the same manner and at the same instances as in theencoding procedure.

The same parallelization as described above with respect to theencoding, is possible at the decoding side. The entropy decoding stageinstantiations 130 shown in FIG. 5 may each be embodied as shown withrespect to the entropy decoding stage 204 in FIG. 6. A probabilityestimation storage 240 may be used in order to store states of theprobability estimations for use in an entropy decoding stage 130responsible for the entropy decoding of a subsequent entropy slice inentropy coding order 16.

After having described embodiments of the present application, theconcepts discussed so far are described in the following, again using adifferent wording. Thereinafter, several further aspects of the presentapplication are described. In the following, the aforementioned codingunits 50 are called LCU (largest coding unit), thereby adapting thewording to the upcoming HEVC standard.

Firstly, the above-discussed probability adaptation 106 is brieflydiscussed again with respect to FIG. 9.

A current LCU uses the probability estimations such as CABACprobabilities, available after coding the preceding left LCU. Forexample, the LCU of FIG. 9, indicated with x, is supposed to be entropycoded using the probability estimation p_(i) as adapted until the end ofentropy coding the left-hand side LCU to the left of LCUx. However, ifthe information is used not only from the left one, but also from one ormore other LCUs, which already have been processed and are available,better results in probability adaptation can be achieved.

As described above, in entropy decoding and entropy encoding the entropyslices, new probability estimations are computed before encoding ordecoding of each LCU, with the help of already existing probabilities(probability estimations) from other LCUs. To be more precise, theprobability adaptation is not only performed from any preceding LCUs ofthe current entropy slice, but also from LCUs of entropy slicespreceding an entropy slice order. This approach is represented in FIG. 9again. The start probability of the current LCU which is denoted X inFIG. 9, can be computed accordingly:p _(new) =a ₁ ·p ₁ +a ₂ ·p ₂ +a ₃ ·p ₃+_(a4) ·p ₄ +a ₅ ·p ₅ +a ₍₅₊₁₎ ·p₍₅₊₁₎ . . . +a _(k) ·p _(k),  (1)where a₁, . . . a_(k) are the weighting factors of the LCUs.

It has been tested which weighting of probabilities provide the bestresults. In this experiment only the neighboring LCUs have been used.The study reveals to use the weighting: 75% from the left LCU and 25%from the above-right LCU. In FIGS. 10-13, the results are presented. Thegraphs entitled using “prob. Adapt.” use the probability adaptationdescribed above.

However, in the probability estimation adaptation, not only adjacentblocks can be used. Each nearest LCU has peculiarly own neighbors, whoseuse for probability optimization may be significant. In other words, notonly the LCUs from the upper nearest row can be applied. In FIG. 9, onecan see an example where the derivations of the probability estimationshave been taken first from the neighbors and can be taken from eachabove-right LCU of each next upper row, cf. p₅ and p_(k).

It should be admitted that some complexity is introduced by the aboveoutlined probability recalculation or probability estimation adaptation.New computation of probability estimation occurs, for example, in threesteps: at first, probability estimations of each candidate have to begained from each context state. This is done by storing in storage 120and 240, respectively, or by steering the parallel entropy slicen/decoding processes such that these states are available concurrently.Secondly, by using the equation (1), optimized probability (p_(new))will be created. That is, some averaging, for example, may be used inorder to combine adapted probability estimations of different entropyslices. As a last step, a new context state is converted from p_(new)and replaces the old one. That is, probability estimation manager 76adopts the new probability estimations thus obtained. This performancefor each syntax element, particularly using multiplication operations,can increase complexity enormously. The only way to reduce this penaltyis to try to avoid these three steps. If the number of candidates andtheir weightings are determined, a pre-calculated table for eachsituation can be approximated. Thus, only one simple access to the tabledates with the help of candidates' indices (context states) is needed.

It is asserted that this technique can provide good results for bothapplications using and not using entropy slices. The first applicationuses only one slice per frame, thus probability adaptation is optimizedwithout any other changes. In case of entropy slices, the adaptation ofthe probabilities occurs within each slice independent from otherslices. This allows for fast learning of the probabilities of thecurrent LCU.

In the above description, the usage of the second LCU of the upper linewas also presented, i.e. the usage of the second LCU for probabilityestimation initialization. Parallelization of encoding and decoding ispossible, if some conditions of the bitstream that have been mentionedabove (entropy slices) are accomplished. The CABAC probabilitiesdependency between LCUs must be broken. By the Wavefront ParallelProcessing is important to make the first LCU of each line independentfrom the last LCU of the previous line. It can be achieved if, forinstance, the CABAC probabilities are re-initialized at the beginning ofeach line of LCUs. However, this method is not optimal, because eachre-initialization loses the achieved CABAC probabilities, which areadapted to the specificities of the image. This penalty can be reduced,if the initialization of the CABAC probabilities of the first LCU ofeach line is occurred with the probabilities obtained after the secondLCU of the previous line.

As has been described above, a gain in probability adaptation speed isachievable by coupling the probability adaptations of spatiallyneighboring entropy slices. In particular, in other words, abovediscussion also anticipates a decoder such as the one of FIG. 6 forreconstructing a sample array (10) from an entropy encoded data stream,configured to entropy decode (performed by the entropy decoding stage) aplurality of entropy slices within the entropy encoder data stream so asto reconstruct different portions (12) of the sample array associatedwith the entropy slices, respectively, with performing, for each entropyslice, the entropy decoding along a respective entropy coding path (14)using respective probability estimations, adapting (performed by theadaptor 78) the respective probability estimations along the respectiveentropy coding path using a previously decoded part of the respectiveentropy slice, starting the entropy decoding the plurality of entropyslices sequentially using an entropy slice order (16), and performing,in entropy decoding a predetermined entropy slice, entropy decoding acurrent part (x) of the predetermined entropy slice based on therespective probability estimations of the predetermined entropy slice asadapted using the previously decoded part of the predetermined entropyslice (including p1, for example), and probability estimations as usedin the entropy decoding of a spatially neighboring, in entropy sliceorder preceding entropy slice (the slice comprising X, for example) at aneighboring part (such as p4) of the spatially neighboring entropyslice.

The different portions may be rows of blocks (e.g. LCUs or macroblocks)of the sample array. The latter may be a picture of a video. The entropycoding path may extend row-wise. The entropy coding, and thus, theprobability adaptation as well, may be context adaptive. The entropyslice order may generally be chosen such that, along the entropy sliceorder, the different portions follow each other in a direction (16)angled relative to the entropy coding paths (14) of the entropy slices,which, in turn, extend substantially in parallel to each other. By thismeasure, a “wavefront” of currently decoded parts (such as p1, p4, p5+1and pk in the figure) of the entropy slices may generally be arrangedalong a line forming an angle to the entropy slice paths smaller thanthe sequence direction of the portions. The wavefront may have to have aslope of 1y per 2x block positions in order for the upper left referenceto be there, for all threads processing slices in parallel.

The decoder may be configured to perform, for each entropy slice, theentropy decoding along the respective entropy coding path in units ofparts of the sample array portion of the respective entropy slice sothat the entropy slices consist of the same number of parts,respectively, and the sequence of parts of the portions along theentropy slice paths are aligned to each other in a direction lateral tothe entropy slice paths. The current part of the portion of thepredetermined entropy slice belongs to the resulting grid of parts (e.g.LCUs or macroblocks). In performing, for each entropy slice, the entropydecoding along a respective entropy coding path, the decoder maypreserve offsets/shifts between starting times between immediatelyconsecutive entropy slices in entropy slice order so that the wavefrontof currently decoded parts of the entropy slices forms a diagonal suchas up to a line with slope of 0.5x block positions with respect toentropy slice paths and entropy slice order direction. The offset/shiftmay correspond to two parts for all pairs of immediately consecutiveentropy slices. Alternatively, the decoder may merely prevent thedistance of currently decoded parts of immediately consecutive (andimmediately neighboring as far as their sample array portions 12 areconcerned) entropy slices to become lower than two parts. See, thefigure above: as soon as the part/block p4 has been decoded, thepart/block to the right thereof is decoded according to path order 16,and concurrently, if any, X is decoded or any of the parts/blockspreceding the same). By this measure, the decoder may use theprobability estimates as already adapted based on the content ofpart/block p4, i.e. the part within the spatially neighboring portion,being aligned to the part of the portion 12 of the predetermined entropyslice succeeding the current part X in path order 16, in order todetermine the probability estimates to be used in decoding X. In case ofa constant offset in decoding the immediately entropy slices of twoparts, the decoder is able to use the probability estimates as alreadyadapted based on the content of part/block p4 concurrently for entropydecoding the subsequent part (i.e. the part to the right of p4) of thespatially neighboring entropy slice.

As described above, a weighted sum of the already adapted probabilityestimations may be used in order to determine the probability estimatesto be used for decoding X.

As also described above, the entropy slice order may also go over frameboundaries.

It should be noted that the just-outlined probability adoption from thepredecessor entropy slices may be performed for every part of thecurrent/predetermined entropy slice for which such neighboring parts inthe predecessor entropy slices are available. That is this is also truefor the first part along the path direction 16, and for this very firstpart/block (the leftmost in each entropy slice in the figure) theadoption equals the initialization described above.

In order to better adapt, also in this case, the 2 above-mentionedmethods can be combined together. The results of this process with 1 and4 Threads, i.e. the parallel used processing units, are illustrated inFIG. 10-FIG. 13 (2LCU+Prob.Adap or 2LCU graphs).

In order to better understand the context of the above embodiments and,especially, the further embodiments described below, in particular theuse of LCUs, one may first have a look at the structure of H.264/AVC.

A coded video sequence in H.264/AVC consists of series of access unitsthat are collected in the NAL unit stream and they use only one sequenceparameter set. Each video sequence can be decoded independently. A codedsequence consists of a sequence of coded pictures. A coded frame can bean entire frame or a single field. Each picture is partitioned intofixed-size macroblocks (in HEVC: LCUs). Several macroblocks or LCUs canbe merged together into one slice. A picture is therefore a collectionof one or more slices. The goal of this data separation is to allowindependent decoding of the samples in the area of the picture, which isrepresented by the slice, without the use of data from other slices.

A technique that has also often referred to as “entropy slices” is asplitting of the traditional slice into additional sub-slices.Specifically, it means slicing of entropy coded data of a single slice.The arrangement of entropy slices in a slice may have differentvarieties. The simplest one is to use each row of LCUs/macroblocks in aframe as one entropy slice. Alternative, columns or separate regions canbe utilized as entropy slices, which even can be interrupted and toggledwith each other, e.g. slice 1 in FIG. 14.

An obvious aim of the entropy slice concept is to enable the use ofparallel CPU/GPU and multi-core architectures in order to improve thetime of the decoding process, i.e. to speed-up the process. The currentslice can be divided into partitions that can be parsed andreconstructed without reference to other slice data. Although a coupleof advantages can be achieved with the entropy slice approach, therebyemerging some penalties.

First of all, a main aim is to create a bitstream that is suitable for aparallel encoding and decoding process. One must take into account, thata LCU can be encoded only if the neighboring LCUs (left, top, top-right)are already available as coded version, in order to use spatial andmotion information for prediction. To enable a parallelism by slicing, ashift between processing of slices must be realized (for example shiftof 2 LCUs, as typical for the wavefront approach). Due to CABACprobabilities adaptation, an LCU use the probabilities that areavailable from the previous encoded LCU. In regard to raster scan order,the problem, which occurs by slicing the picture, is the prevention ofparallelism, since the first LCU of each line depends on the last LCU ofthe previous line. On impact of this is that the CABAC probabilitiesdependencies between slices must be broken, so that several slices canbe started simultaneously. One way to do this is the usual CABACre-initialization whereby, however, all the adopted data will be lost.As a result, bitrate may increase.

Secondly, each slice generates its own sub-bitstream, which can be putin series into the main stream. But it is necessitated to pass decoderspecial information, so that these slices and their positions in themain stream can be correctly identified. Two scenarios for signaling aresupported. The location information can be stored in the picture header(slices length information) or in each slice header (points as startcode). Byte alignment at the end of each entropy slice and locationinformation raise loss.

In order to reduce the penalty introduced by the signaling for entropyslices, it is essential to use a good coding technique for thesignaling. A noticeable penalty for entropy slice signaling isintroduced in a frame, if start codes are used for each slice, i.e. toomuch additional bytes (e.g. min. 4 bytes per slice) are added to thebitstream. Certainly, insertion of entropy slices 90 using start codeshas the advantage in low delay scenarios where the encoder shall be ableto instantaneously output entropy slices. In such cases, there is noup-front signaling of entry points possible.

In less low-delay strict scenarios, however, the opportunity to storeslice lengths (offsets) seems more suitable. One well known method tocode such information is Variable Length Code (VLC) orExponential-Golomb coding. The main feature of VLC is adding emptyinformation (zeros) before true information. With the help of thesezeros, a code that stores information about offset length can bedetermined. We propose another technique to realize this, which schemais shown in FIG. 15, where X=EntropySliceSize is the amount of bytescontained in an entropy slice. Each next X (offset) is defined as adifference in the size between previous, already encoded and signaled,entropy slice offset and the current one. The main features of thisconcept are forming of differences, depending on size X, thus the amountof data can be diminished, and adding up to 3 bits, which allowsextracting the proper information about size of each entropy slice bydecoder. In comparison with VLC the saving bit quantity in the entropyslice header can be gained.

That is, in accordance with the aspect of FIG. 15, for example, aconcept for entropy encoding an information signal is provided whichenables higher compression rate despite parallel processing ability thancompared to concepts available so far. In accordance with this aspect,an entropy encoded data stream section 20 into which a sample array isencoded, comprises entropy slices such as 90 in FIG. 8 into whichdifferent portions 12 of the sample array are entropy encoded, and aheader illustrated with dashed lines 300 in FIG. 9, comprisinginformation which reveals starting positions 302 of the entropy slices90, measured in the entropy decoded domain, within the entropy-encodeddata-stream, the information comprising, for a predetermined entropyslice, a difference value revealing a difference between a startingposition of a preceding entropy slice n−1 and a starting position of thepredetermined entropy slice n, the difference being included into thedata stream as a VLC bit sequence.

In particular, the VLC bit sequence may have a prefix of variable lengthy, indicating that the difference x falls into the y-th of a sequence ofa number z of intervals [0,2^(a)−1], [2^(a),2^(b)+2^(a)−1],[2^(b)+2^(a), 2^(c)+2^(b)+2^(a)−1] as so forth, and a PCM coded suffixof the y-th of the sequence of lengths a, b, c . . . If a, b, c, . . .are chosen to be a power of two, if the corresponding y is added, i.e.so that a+1, b+2, c+3 and so forth are all powers of two, then bytealignment may be preserved. The number z is not restricted to three asexemplarily chosen in FIG. 15.

The encoder, such as the one in FIG. 1, is appropriately designed toturn the difference of consecutive starting positions into a VLC bitsequence, i.e. by firstly determining the prefix, i.e. into whichsub-interval the difference lies (the y-th), and then setting the suffixplus to be equal to the starting position difference minus the y-th of0, 2^(a), 2^(b)+2^(a) as so forth. The decoder, such as in FIG. 6, isappropriately designed to derive the starting position of the currententropy slice n from the VLC bit sequence, i.e. by firstly inspectingthe prefix to obtain y, then setting the difference to the value of thesuffix plus the y-th of 0, 2^(a), 2^(b)+2^(a) as so forth, and thenadding the difference to the starting point of the preceding entropyslice n−1.

A further advantage may be achieved by segmentation of entropy slices,namely for low-delay transmission and decoding speed-up.

In video-streaming, enabling of higher resolutions (Full-HD, QUAD-HDetc.) leads to higher amount of data that has to be transmitted. Fortime-sensitive scenarios, so called Low-Delay use-case (<145 ms), thetransmission time becomes a critical factor. Consider the up-load linkof ADSL for a video conferencing application. Here, so called randomaccess points of stream, usually these refer to I-frames, will be thecandidates to cause a bottleneck during transmission.

To solve that problem and to minimize the delay of transmission anddecoding time, i.e. the end-to-end delay, a new technique for aninterleaved entropy slice scheme for parallel transmission andprocessing may be applied.

HEVC will allow for so called Wavefront-processing at the decoder side.That is going to be enabled by use of entropy slices. In normal case thedata of a whole slice will be delivered at one time. The decodingprocess will start as soon as encoded data is arrived atWavefront-decoder-engines. To reduce the time when decoder can start andfinish a frame, a segmentation of entropy slices into small chunks usingan interleaving approach is used in accordance with the presentembodiment. Hence the encoder can deliver data, correspondent toparticular entropy slice, to transport layer earlier than in normalcase. It results then in faster transmission and earlier start of aparallel decoding process at the client.

Creating chunks of a slice can be further achieved by sub-dividing anentropy slice in further slices keeping all dependencies (dependentslices). If done so in each Largest Coding Unit (LCU)/Coding tree unit(CTU), these chunks can be interleaved by additionally using systemlayer techniques which allow for transmitting the chunks in aninterleaved form and recovering or at least providing the knowledge ofthe original decoding order of the chunks via additional signaling. Suchsignaling may be a decoding order number (DON), such as defined in theIETF RTP Payload Format for H.264/AVC (RFC 3984). Another system methodmay be to apply the chunks of a wavefront substream to a differenttransport stream, as in MEPG-2 Systems, by assigning a different PID toeach of them, further multiplexing and by that interleaving them on thetransport channel.

The approach may be also applied over frame boundaries, in case, if thefollowing frame's slice(s) or entropy slice(s) can be already decoded,e.g. in wavefront manner, based on the knowledge that the necessitatedinformation for decoding an entropy slice of a following frame due tothe availability of inter-frame references. Those already decodable dataof a frame succeeding in decoding order may be derived from the maximumallowed/signaled motion vector length or additional information in thestream indicating the dependencies of data parts to the precedingframe(s)), or a fixed referencing scheme, indicating the positionsignaled in a sequence-fixed position such as a parameter set. This willfurther be outlined below.

Picture may be encoded with one entropy slice per largest coding unit(LCU)-row(s) as depicted in FIGS. 2 and 9. It is advantageous for usingof Wavefront-technique at decoder side. During the encoding process,bitstream of each slice may be divided into segments of constant size.Then the resulted segments are interleaved and can be passed to thetransmission. The constant size of segments can bring a problem at theend of bitstream due to its variable length.

There are two general solutions possible. First one is a generation ofone-byte segments (usually the bitstream representation of slice isbyte-aligned) and controlling of consuming of bytes by eachdecoder-engine, i.e. the decoder finds out the completion of an entropyslice.

The second is using of finalizing code at the end of slice. In the lastcase, variable length segments are possible, but it can also lead tohigher amount of data. Another method is the signaling of the entropyslice length. One way of such an alternative is described below

The size of segment and interleaving mode can be signaled either in oneSEI-Message or in SPS.

The transmission scheme is shown in FIG. 16.

Thus, in accordance with the aspect of FIG. 16, for example, a conceptfor entropy encoding a sample array is provided which enables lowerdelay than compared to concepts available so far. In accordance withthis aspect, an encoded data stream 20 into which an information signalis encoded, comprises slices such as entropy slices 90 or just fullyindependently decodable slices (shown at the left-hand side) into whichdifferent portions 12 of the information signal are (predictively and/orentropy) encoded, wherein the slices 90 are sub-divided into chunks(shaded boxes 310) which are arranged within the encoded data stream 20in an interleaved manner (shown at the right-hand side), theinterleaving being represented by bracket 312.

As denoted above and described with respect to the other aspects, theslices may be entropy slices 90 which, in turn, may be proper subsets ofslices of frames, and accordingly, the encoded data stream may be anentropy encoded data stream 20.

The interleaving 312 of the slices 90 enables lower delay since thedecoder responsible for decoding any of the slices 90 does not need towait for a time duration consumed by previous slices of other decoders(according to slice order 16). Rather, all available decoders are ableto begin decoding their associated slices as soon as the first chunk 310thereof is available and potential inter-slice dependencies areresolved, cf. wavefront approach.

The different portions 12 may be entropy encoded into the entropy slices90 using probability estimations which are set independently among theentropy slices, or using inter-entropy-slice adoption of probabilityestimates as described above such as, for example, namely in that thedifferent portions 12 are encoded into the entropy slices 90 alongrespective entropy coding paths 14, and subordinate entropy slices havetheir corresponding portions 12 encoded therein using probabilitiesestimates derived, inter alias, from probability estimates used inhigher ranking entropy slices at spatially neighboring parts of thecorresponding portion, as described previously.

The entropy encoded data stream 20 may additionally comprise a header asis shown as an option in FIG. 16. The header 300 may be associated withthe frame (sample array) of a sequence 30, the header 300 comprisinginformation which reveals a length of the entropy slices. Theinformation regarding the length of the entropy slices 90 may be codedwithin the header 300 as described above using VLC codes. Using theknowledge about the length of the entropy slices, at the decoding side,it is possible to identify a last chunk associated with each of theentropy slices 90 and the length thereof. However, startcodes or otherindication schemes may also be used. The slice's start positions may bealso identified just by the decoding process which knows the terminationof a slice. Therefore it may be possible to just rely on the indicationof the decoder, but this necessitates signaling between decoders and insome cases, if an “earlier” entropy slice terminates later than a“later” slice in the stream. This may necessitate for the specific casesan “adaptive” signaling in the stream, which may be based on startcodes.

For example, the entropy slices 90 may be arranged after the header 300as illustrated in FIG. 16.

The chunks 310 may be of equal length, at least as far as a beginningcontiguous portion of the sequence of chunks 310 from a first one of thechunks on along an order in which the chunks are arranged within theentropy encoded data stream 20, is concerned. Subsequent chunks may varyin length. The subsequent chunks may be equal to the length of thebeginning contiguous portion of the sequence or smaller than thislength. The length of the subsequent chunks may be derived from theaforementioned information within the header 300 revealing the length ofthe entropy slices 90 or the start position.

The chunks 310 may be arranged within the entropy encoded data stream 20in a cyclic manner in accordance with an order defined among the entropyslices. For entropy slices, the chunks of which are completely withinthe previous cycles, may be skipped in the current and subsequentcycles.

Other information signals then a sample array sequence such as a videosignal could by conveyed via data stream 20 as well. The differentportions 12 need, thus, be no portions of a predetermined sample arraysuch as a picture/frame.

As described above, the entropy slices 90 may have the differentportions 12 of the sample array 10 encoded thereinto via predictivecoding using inter-entropy-slice prediction and/or inter-frameprediction an entropy encoding of a prediction residual of theinter-entropy-slice prediction and/or inter-frame prediction. That is,as described above, the different portions may be spatially distinctportions of the frame 10 or multiple frames 30. Latter case applies, ifthe following frame's(s') slice(s) or entropy slice(s) can be alreadydecoded, e.g. in wavefront manner, based on the knowledge that thenecessitated information for decoding an entropy slice of a followingframe due to the available inter-frame references. Those alreadydecodable data of a frame succeeding in decoding order may be derivedfrom the maximum allowed/signaled motion vector length or additionalinformation in the stream indicating the dependencies of data parts tothe preceding frame(s)), and inter-entropy-slice prediction may involveintra prediction, while inter-frame prediction may involvemotion-compensated prediction. An example is outlined below.

The aforementioned independency of the probability estimation settingamong the entropy slices, may relate to both probability adaptation aswell as context modeling. That is, the context chosen within an entropyslice, may be chosen independently from other entropy slices, and theprobability estimation of a context may be initialized and adaptedindependent from any other entropy slice as well.

An corresponding entropy decoder may be constructed as follows.

An entropy decoder configured to decode an entropy encoded data stream20 which comprises entropy slices 90 into which different portions of aframe are entropy encoded, wherein the entropy slices are sub-dividedinto chunks 310 which are arranged within the entropy encoded datastream 20 in an interleaved manner, may be configured as shown in FIG. 6and may additionally comprise a de-interleaver configured tode-interleave the chunks 310, symbolized with 314 in FIG. 16.

In particular, as illustrated in FIG. 5, the entropy decoder maycomprise a plurality of entropy decoders 130 such as threads running ondifferent processing kernels, wherein the de-interleaver may beconfigured to, for each entropy slice, forward the chunks 310 thereof toan entropy decoder 44 associated with the respective entropy slice.

In other words, the entropy slices may be sub-divided into chunks,which, in turn, may be interleaved, and the decoder may comprise ade-interleaver to de-interleave the chunks and may start operating onthe entropy slices in parallel along the paths 16 even before thereception of any of the entropy slices as a whole. It should be recalledthat the length of the chunks may be measured in the entropy encodeddomain rather than in the syntax domain so as to correspond, forexample, to a number of certain spatial parts/blocks in the picture orthe like although the latter option would be available as well.

Next, possible usage of temporal dependencies is described. Same may beused in addition to, or alternatively to the probability estimationenhancement embodiments described so far.

Wavefront Processing pattern may, as it is described now, be extended tothe entropy coding with the new probabilities calculation for each LCUto use also the temporal dependencies between frames.

As well-known, the probabilities will be re-initialized at the beginningof each frame (the first LCU). Thereby the probabilities, which havealready been acquired in the previous frame, are lost. In order todecrease the lose of coding efficiency, one may pass the end state ofpicture (cp. 320) or, in the case of entropy slice usage, end state ofslice (cp. 322) of the probabilities from the reference frame 324 to thefirst LCU 50 of the current frame 10 or entropy slice 12 respectively(FIG. 17). Such data of a corresponding slice in a reference frame maybe derived not only at the end position but also from at a priorposition in the referenced slices, since the parallel wavefrontprocessing may also go over frame boundaries, i.e. while coding a sliceof a frame, the coding process of the slice of the preceding frame maynot already be finished. Therefore, signaling may be used to indicatethe reference position, or may be indicated by scheme.

Using above notation, one may thus, for initializing in starting phase84 and 234, respectively, set P_({i,j}) with {i,j} denoting the first CU50 in the k-th entropy slice of the current sample array equal to, or atleast depending on, any T(P_({i,j}′)) with {i,j}′ denoting a CU within apreceding (in sample array coding order which may be equal to thepresentation order) sample array or a combination of severalT(P_({i,j}′))'s. This may be done for k=0 only, or for each entropyslice kϵ{1 . . . K} with K denoting the number of entropy slices in thecurrent frame. The temporal initialization may be done additionally oralternatively to the spatial initialization described above. That is,P_({i,j}) with {i,j} denoting the first CU 50 in the k-th entropy slicemay be set equal to some combination (such as sme average) ofT(P_({i,j′})) and T(P_({i,j}spatial)) with {i j}′ denoting a CU within apreceding (previously de/coded) sample array or a combination of severalT(P{i,j}′)'s and {i,j}_(spatial) denoting a CU within a precedingentropy slice of the current sample array. As far as the location of{i,j}′ is concerned, P_({i,j}) with {i,j} denoting the first (in entropycoding order 14) CU 50 in the k-th (in entropy coding order 14) entropyslice of the current sample array may be set equal to T(P_({i,j}′)) with{i,j}′ denoting the last (in entropy coding order 14) CU within the k-th(in entropy slice order) entropy slice in the preceding (in sample arraycoding order) sample array or the last CU within the last (in entropyslice order) entropy slice in the preceding (in sample array codingorder) sample array. Again, this temporal initialization may beperformed for the first entropy slice in the sample array only.

The parsing process of the end state of the reference frame was testedwith the method of probabilities adaptation, which results areillustrated on FIG. 10-FIG. 19 (Temporal graph).

Another opportunity to use the dates from other frames is to exchangethe obtained probabilities between collocated LCUs. The main idea isbased on assertion that properties of reference frame differ fromcurrent frame not greatly. In order to quicken the learning of theprobabilities along LCUs in a frame, one can try to pass the end stateof each LCU to the appropriate LCU in current frame. This proposal isillustrated on FIG. 18.

Under reference frame, different opportunities can be understood. As forinstance, a frame, which is the last coded can be used as a referenceframe. Otherwise the last coded frame only from the same temporal layercan be appropriated for reference.

Moreover, this approach can be merged with already above proposedmethods, as usage of last (slice) information from reference frame,probabilities adaptation and usage of the second LCU of the upper line.

The above spatial adaptation process may be modified to beP _(p(n+1))=average(T(P _(p(n))),T(P _({i,j}) ₁ ), . . . ,T(P _({i,j})_(N) ),T(P _({i,j}′) ₁ ), . . . ,T(P _({i,j}′) _(M) )₁)where N may be 1 or more than 1 and {i,j}₁ . . . N is/are selected from(lie within) any previous (in entropy slice order 16) entropy slice 90in the current sample array 10 and its associated portion 12,respectively, and M may be 1 or more than 1 and {i,j}′₁ . . . M is/arelie within the preceding sample array 350. It may be that (the) at leastone of CU's 50 {i,j}′₁ . . . M is co-located to p(n). With regard topossible selections of CU's 50 {i,j}₁ . . . N reference is made to theabove description. The function “average” may be one of a weighted sum,a median function, etc.

The above spatial adaptation process may be replaced byP _(p(n+1))=average(T(P _(p(n)) ,T(P _({i,j}′) ₁ ), . . . ,T(P _({i,j}′)_(M) ),)where M may be 1 or more than 1 and {i,j}′₁ . . . M is/are lie withinthe preceding sample array. It may be that (the) at least of {i,j}′₁ . .. M is co-located to p(n). With regard to possible selections of {i,j₁ .. . N reference is made to the above description. The function “average”may be one of a weighted sum, a median function, etc. It may be that(the) at least of {i,j}′₁ . . . M is co-located to p(n).

As a specific extension of usage of collocated information, an approachto utilize the obtained dates from other blocks from one or even morereference frames can be applied.

Earlier mentioned techniques use only the obtained information from thedirect neighbors in current frame or in reference frames. However itdoes not mean that the gained probabilities in this case are the bestones. The adjacent LCUs, according to the picture partitions(residuals), do not have the best probabilities models. It is supposedthat the best results can be achieved with the help of blocks, fromwhich prediction will be done. And thus, this appropriate block can beused as a reference for the current LCU.

Thus, in the above adaptation example, {i,j}₁ . . . N and/or {i,j}′₁ . .. M may be selected depending on CU's serving as providers of predictorsfor p(n).

The temporal probability adaptation/initialization schemes presented canalso be used without entropy slices or a single entropy slice per frame.

In accordance with the latter aspect, a gain in probability adaptationspeed is achieved by coupling the probability adaptations of temporallyneighboring/related frames. What it is described there is, a decodersuch as the one of FIG. 6, wherein the decoder is configured toreconstruct a sequence of sample arrays from an entropy encoder datastream, and is configured to entropy decode a current frame of theentropy encoder data stream so as to reconstruct a current sample arrayof the sequence of sample arrays, perform the entropy decoding along anentropy coding path and using probability estimations and adapt theprobability estimations along the entropy coding path using a previouslydecoded part of the current frame, wherein the entropy decoding stage isconfigured to initialize or determine the probability estimations forthe current frame based on probability estimations used in decoding apreviously decoded frame of the entropy encoded data stream.

That is, for example, the probability estimations for the current frameare initialized based on probability estimations resulting after havingfinished decoding the previously decoded frame of the entropy encodeddata stream. Buffer requirements are, therefore low, since merely theend state of the probability estimates have to be buffered till thestart of the decoding of the current frame. Of course, this aspect iscombinable with the aspect of FIGS. 1 to 9 in that for the first partsof each portion 12 not only probability estimates used for spatiallyneighboring parts in prior entropy slices (if available) but also, in aweighted manner, for example, the end state of the probability estimatesof a (for example, spatially) corresponding entropy slice in theprevious frame is used. Such data of a corresponding slice in areference frame may be derived not only at the end position but alsofrom a prior position in the referenced slices, since the parallelwavefront processing may also go over frame boundaries, i.e. whilecoding a slice of a frame, the coding process of the slice of thepreceding frame may not already be finished. Therefore, signaling may beused to indicate the reference position, or may be indicated by scheme.

Further, for example, the probability estimations used for coding theparts/blocks of the previously decoded frame are buffered all, not onlythe end state, and the decoder would, in entropy decoding thepredetermined entropy slice (with reference to above description ofspatially couple probability derivation), perform entropy decoding thecurrent part (X) of the predetermined entropy slice based on therespective probability estimations of the predetermined entropy slice asadapted using the previously decoded part of the predetermined entropyslice (including p1, for example), and probability estimations as usedin the entropy decoding of a spatially corresponding part of an entropyslice of the previously decoded frame with, optionally, additionallyusing probability estimations as used in the entropy decoding of aspatially neighboring, in entropy slice order preceding entropy slice(the slice comprising X, for example) at a neighboring part (such as p4)of the spatially neighboring entropy slice, as described above. As hasalso been described above, the spatial correspondence among parts, andthe identification of an appropriate one for probability adoption forthe current frame among the previously decoded frame may be defined byhelp of motion information such as, motion indices, motion vectors andthe like, of the current part/block.

Up to now, the wavefront extending during wavefront processing hasprimarily been described as extending obliquely through one sample array10 with the coding/decoding being performed one sample array after theother. However, this is not a must. Reference is made to FIG. 19. FIG.19 shows a portion out of a sequence of sample arrays, wherein thesample arrays of the sequence have defined thereamong, and are depictedas being arranged in, a sample array coding order 380 which may or maynot coincide with a presentation time order. FIG. 19 exemplarily shows asubdivision of the sample arrays 10 into four entropy slices each.Already coded/decoded entropy slices are shown hatched. Fourcoding/decoding threads (coding/decoding stages) 382 are currentlyoperating on the four entropy slices 12 of sample array having index n.However, FIG. 19 shows a thread number 5 being left, and it is possiblethat this further thread 382 having number 5 in FIG. 19 operates tocode/decode the next sample array in line, i.e. n+1, at portions forwhich it is guaranteed that respective reference portions in currentlycoded/decoded frame n are already available, i.e. have already beenprocessed by any of threads 1 to 4. These portions are referenced inpredictions shown at 64 in FIG. 1, for example.

FIG. 19 exemplarily shows with a dotted line 384, a line extendingthrough sample array n+1 which is co-located to the border between thealready processed, i.e. already coded/decoded portion of sample array n,i.e. the hatched portion within sample array n on the one hand, and thenon yet processed portion, i.e. the un-hatched portion of sample array non the other hand. With double headed arrows, FIG. 19 also shows themaximum possible length of motion vectors measured in column and rowdirection, i.e. y_(max) and x_(max), respectively. Accordingly, FIG. 19also shows with a dash-dotted line 386 a displaced version of line 384,namely a line 386 which is spaced apart from line 384 at the minimumpossible distance so that the distance does not fall below y_(max) incolumn direction and x_(max) in row direction. As can be seen, there arecoding units 50 in sample array n+1 for which any reference portion insample array n is guaranteed to be found as being completely containedwithin the already processed portion of this sample array n, namelythose laying in the half of sample array n+1 lying on the upstream siderelative to line 386. Accordingly, thread 5 is able to already operateto decode/code these coding units as shown in FIG. 19. As can be seen,even a sixth thread could operate on the second entropy slice in entropyslice order 16 of sample array n+1. Thereby, the wavefront extends notonly spatially but also temporally through the spatio-temporal spacespanned by the sequence 30 of sample arrays.

Please note that the just-mentioned wavefront aspect does also work incombination with the above-outlined probability estimation couplingsacross entropy slice boarders.

Further, with respect to the above outlined chunk aspect, it should alsobe noted that subdividing entropy slices into smaller pieces, i.e.chunks, is not restricted to being performed in the entropy codeddomain, i.e. in the entropy compressed domain. Think of the abovediscussion: entropy slices as described above have the advantage ofreducing the coding efficiency loss despite the enablement of wavefrontprocessing due to the derivation of probability estimations frompreviously coded/decoded entropy slices of the same or a previouslycoded/decoded frame, i.e. an initialization and/or adaptation of theprobability estimations based on probability estimations of suchprevious entropy slices. Each of these entropy slices is supposed to beentropy coded/decoded by one thread in case of wavefront processing.That is, when subdividing the entropy slices, it is not necessary torender the chunks codable/decodable in parallel. Rather, the encodershall merely be provided with the opportunity to output subparts of itsentropy slice's bitstream prior to finalizing the entropy encoding, andthe decoder shall be provided with the opportunity to operate on thesesubparts, i.e. chunks, prior to the reception of the remaining chunks ofthe same entropy slice. Moreover, the interleaving shall be enabled atthe reception side. In order to enable the latter de-interleaving,however, it is not necessary to perform the subdivision in the entropycoded domain. In particular, it is possible to perform the abovepresented subdivision of entropy slices into smaller chunks withoutsevere coding efficiency loss, by intermittently merely resetting theinternal state of the probability interval, i.e. the probabilityinterval width value, and the offset value, respectively, of the entropycoding/decoding core. The probability estimations, however, are notreset. Rather, they are continuously updated/adapted from the beginningto the end of the entropy slices, respectively. By this measure, it ispossible to subdivide the entropy slices into individual chunks with thesubdivision being performed in the syntax element domain rather than thecompressed bitstream domain. The sub-division may follow a spatialsub-division as outlined below in order to ease a signaling of the chunkinterfaces to the decoder. Each of the chunks could be provided with itsown chunk header revealing, for example, its starting position in thesample array, measured, for example, with respect to the coding order 14relative to the respective entropy slice's start position along with anindex to its entropy slice, or relative to a prominent location of thesample array 10 such as the upper left corner.

In order to more clearly describe the subdivision of entropy slices intochunks in accordance with the latter embodiment, reference is made toFIG. 20. FIG. 20 shows, merely for illustrative purposes, sample array10 as being subdivided into four entropy slices. Currently codedportions of sample array 10 are shown hatched. Three threads arecurrently operating on the entropy encoding of sample array 10 andoutput chunks of the entropy slices on an immediate-attention basis: seefor example the first entropy slice in entropy slice order 16 whichcorresponds to portion 12 of sample array 10. After having encoded asubpart 12 a of portion 12, the encoder forms a chunk 390 therefrom,i.e. the entropy coding core 80 performs some finishing procedure tofinalize the arithmetic bitstream produced from sub-portion 12 a so farin case of arithmetic coding to form chunk 390. The encoding procedureis then resumed with respect to the subsequent sub-portion 12 b ofentropy slice 12 in coding order 14 while starting a new entropybitstream. This means, for example, that internal states, such asprobability interval width value and probability interval offset valueof entropy coding core 80, are reset. The probability estimations,however, are not reset. They are left unchanged. This is illustrated inFIG. 20 by an arrow 392. In FIG. 20, it is exemplarily shown that theentropy slice or portion 12 is subdivided into more than twosub-portions, and accordingly even the second chunk 1 b is subject tosome entropy finishing before reaching the end of portion 12 alongcoding order 14, whereupon a next chunk in line is started with etc.

Concurrently, another thread operates on the second entropy slice orportion 12 in entropy slice order 16. Upon finishing a first sub-portionof this second entropy slice/portion 12, a chunk 2 a is output,whereupon entropy coding the remainder of the second entropy slice iscommenced while maintaining, however, the probability estimation asvalid at the end of chunk 2 a.

With a time axis 394, FIG. 20 seeks to illustrate that the chunks 390are output as soon as they have been finalized. This leads to aninterleaving similar to the one depicted in FIG. 16. Each chunk may bepacketized into a packet and transported to the decoding side via sometransport layer in any order. The transport layer is illustrated usingarrow 396.

The decoder has to reassign the chunks to its sub-portions 12 a, 12 band so forth. To this end, each chunk 390 may have a header section 398revealing the location of the beginning of its associated sub-portion 12a or 12 b, i.e. the sub-portion the syntax elements describing the sameare entropy coded in the respective chunk. By using this information,the decoder is able to associate each chunk 390 with its entropy sliceand with its sub-portion within the portion 12 of that entropy slice.

For illustration purposes, FIG. 20 also exemplarily shows thepossibility that the junction between consecutive sub-portions 12 a and12 b of an entropy slice 12 does not have to coincide with the borderbetween consecutive coding units 50. Rather, the junction may be definedin a deeper level of the aforementioned exemplary multi-tree subdivisionof the coding units. The location information contained in headers 398may indicate the beginning of the sub-portion associated with thecurrent chunk 390 precisely enough in order to identify the respectivesub-block of the respective coding unit, i.e. the location within thesyntax element sequence from which on the respective sub-block isdescribed.

As became clear from the above discussion, almost no coding efficiencyloss resulted from the subdivision of the entropy slices into chunks.Merely the entropy finishing processes and the packetizing may involvesome coding efficiency loss, but on the other hand low delay gains areenormous.

Again, please note that the just-mentioned spatial sub-division chunkaspect does also work in combination with the above-outlined probabilityestimation couplings across entropy slice boarders, spatially andtemporally.

The decoder such as decoder of FIG. 6, may undo the chunk transmissionas follows. IN particular, the decoder may check as to which entropyslice a current chunk belongs to. This check may be done based on theafore-mentioned location information. Then, it may be checked whetherthe current chunk corresponds to a first sub-portion of the portion ofthe corresponding entropy slice along the entropy coding path 14. If so,the decoder may entropy decode the current chunk under adapting therespective probability estimations and take a state of the respectiveprobability estimations as manifesting themselves at an end of entropydecoding the current chunk into account when entropy decoding anotherchunk which corresponds to a second sub-portion of the portion of thepredetermined entropy slice along the entropy coding path. The “takinginto account” may involve setting the probability estimations at thebeginning of chunk 1 b equal to the probability estimations manifestingthemselves, by the probability adaption starting from the probabilityestimation state at the beginning of chunk 1 a, at the end of chunk's 1a sub-portion 12 a, or equal to a combination thereof with entropyprobability estimations from other entropy slices as described above. Asfar as the probability initialization at the beginning of the firstchunk 12 a is concerned, reference is made to the above discussion, asthis also forms the beginning of the corresponding entropy slice. Inother words, if the current slice is a second or following chunk inorder 14, decoder may entropy decode the current chunk using probabilityestimations which depend on probability estimations manifestingthemselves at an end of entropy decoding a chunk which corresponds to asub-portion of the portion of the predetermined entropy slice precedingthe sub-portion corresponding to the current chunk, along the entropycoding path 14.

The above description reveals different methods, which can be useful forparallel encoding and decoding as well as helpful for optimization ofalready existing processes in the emerging HEVC video coding standard. Ashort overview of entropy slices has been presented. It has been shownhow they can be formed, which advantages can be achieved by slicing andwhat penalties can occur from those techniques. A number of methods havebeen proposed, which are supposed to improve the learning process ofprobabilities along LCUs (largest coding unit) in frame, by betterexploiting the local dependences between LCUs, and also the temporaldependences between LCUs of different frames. It is asserted thatdifferent combinations provide improvements for both concepts with andwithout parallelization of encoding and decoding.

The performance improvement in High Efficiency, for instance, by thebest mix of proposed approaches, is −0.4% in Intra, −0.78% in Low Delayand −0.63% in Random Access in comparison with HM3.0 without use ofentropy slices or −0.7% in Intra, −1.95% in Low Delay and −1.5% inRandom Access in comparison with entropy slice approach with usualre-initialization.

In particular, inter alias, the following techniques have been presentedabove.

to use not only local but also temporal dependencies of LCUs, tooptimize adaptation of the CABAC probabilities before coding each LCU,see FIGS. 1 to 9, 17 and 18.

to achieve more flexibility in decoding, also entropy slices can beutilized, so that certain regions in frame become independent from eachother.

to allow minimal signaling of the slice/entropy slices start positionsfor parallelized, e.g. wavefront processing, see FIG. 15

to allow low delay transport in an parallelizedencoder-transmitter-receiver-decoder environment through interleavedtransport of entropy slices/slices, see FIG. 16.

All the methods, which were mentioned above, have been integrated andtested in HM3.0.

The obtained results, where reference point is HM3.0 without any entropyslice implementation, are presented in tables 1 and 2 (where 2LCU-usageof the second LCU of the upper line; 2LCU+Prob.Adap-2LCU merged with themethod of probabilities adaptation; Temporal--usage of temporaldependences (end state of a reference frame) with the probabilitiesadaptation for each LCU).

TABLE 1 Summary RD results with 1 Thread 2LCU + Tem- 1 Thread 2LCUProb.Adap Prob.Adap poral Intra 0.14 −0.31 −0.4 LowDelay 0.66 0.09 −0.12−0.78 Random 0.48 −0.1 −0.24 −0.63

TABLE 2 Summary RD results with 4 Threads 2LCU + Tem- 4 Threads 2LCUProb.Adap Prob.Adap poral Intra 0.19 −0.27 −0.27 LowDelay 1.01 0.54 0.630.17 Random 0.74 0.2 0.24 0.01

However, it is interesting to know how the proposed approaches affectthe wavefront processing with the re-initialization of probabilities atthe beginning of each line of LCUs. These results are illustrated ontables 3 and 4 (where orig_neilnit is comparison HM3.0 without use ofentropy slices with usage of entropy slices with re-initialization).

TABLE 3 Summary RD results with 1 Thread. Reference is newinitialization. 2LCU + Tem- 1Thread orig_neilnit 2LCU ProbAdap ProbAdaptporal Intra 0.3 −0.15 −0.62 −0.7 RandomAccess 0.9 −0.4 −0.99 −1.13 −1.51LowDelay 1.19 −0.51 −1.08 −1.29 −1.95

TABLE 4 Summary RD results with 4 Threads. Reference is newinitialization 2LCU + Tem- 4Threads orig_neilnit 2LCU ProbAdap ProbAdaptporal Intra 0.34 −0.14 −0.61 −0.61 RandomAccess 1.18 −0.43 −0.96 −0.96−1.15 LowDelay 1.61 −0.5 −0.92 −1.05 −1.41

The above results show that considerably more use of dependences withinand between frames and rational application of already obtainedinformation prevent average loss.

An approach for Wavefront Processing for HEVC video encoding anddecoding merges the possibility to use dependences between adjacent LCUsas well as temporal frame dependences with the concept of WavefrontParallel Processing. In this way the loss can be reduced and theperformance advance can be achieved.

A gain in probability adaptation speed has been achieved by computingthe probability adaptations of spatially neighboring entropy slices.

As already mentioned above, all of the above aspects may be combinedwith each other, and thus, the mentioning of certain implementationpossibilities with regard to a certain aspect shall, of course, alsoapply to the other aspects.

Although some aspects have been described in the context of anapparatus, it is clear that these aspects also represent a descriptionof the corresponding method, where a block or device corresponds to amethod step or a feature of a method step. Analogously, aspectsdescribed in the context of a method step also represent a descriptionof a corresponding block or item or feature of a correspondingapparatus. Some or all of the method steps may be executed by (or using)a hardware apparatus, like for example, a microprocessor, a programmablecomputer or an electronic circuit. In some embodiments, some one or moreof the most important method steps may be executed by such an apparatus.

The inventive encoded signals mentioned above can be stored on a digitalstorage medium or can be transmitted on a transmission medium such as awireless transmission medium or a wired transmission medium such as theInternet.

Depending on certain implementation requirements, embodiments of theinvention can be implemented in hardware or in software. Theimplementation can be performed using a digital storage medium, forexample a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM,an EEPROM or a FLASH memory, having electronically readable controlsignals stored thereon, which cooperate (or are capable of cooperating)with a programmable computer system such that the respective method isperformed. Therefore, the digital storage medium may be computerreadable.

Some embodiments according to the invention comprise a data carrierhaving electronically readable control signals, which are capable ofcooperating with a programmable computer system, such that one of themethods described herein is performed.

Generally, embodiments of the present invention can be implemented as acomputer program product with a program code, the program code beingoperative for performing one of the methods when the computer programproduct runs on a computer. The program code may for example be storedon a machine readable carrier.

Other embodiments comprise the computer program for performing one ofthe methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the inventive method is, therefore, acomputer program having a program code for performing one of the methodsdescribed herein, when the computer program runs on a computer.

A further embodiment of the inventive methods is, therefore, a datacarrier (or a digital storage medium, or a computer-readable medium)comprising, recorded thereon, the computer program for performing one ofthe methods described herein. The data carrier, the digital storagemedium or the recorded medium are typically tangible and/ornon-transitionary.

A further embodiment of the inventive method is, therefore, a datastream or a sequence of signals representing the computer program forperforming one of the methods described herein. The data stream or thesequence of signals may for example be configured to be transferred viaa data communication connection, for example via the Internet.

A further embodiment comprises a processing means, for example acomputer, or a programmable logic device, configured to or adapted toperform one of the methods described herein.

A further embodiment comprises a computer having installed thereon thecomputer program for performing one of the methods described herein.

A further embodiment according to the invention comprises an apparatusor a system configured to transfer (for example, electronically oroptically) a computer program for performing one of the methodsdescribed herein to a receiver. The receiver may, for example, be acomputer, a mobile device, a memory device or the like. The apparatus orsystem may, for example, comprise a file server for transferring thecomputer program to the receiver.

In some embodiments, a programmable logic device (for example a fieldprogrammable gate array) may be used to perform some or all of thefunctionalities of the methods described herein. In some embodiments, afield programmable gate array may cooperate with a microprocessor inorder to perform one of the methods described herein. Generally, themethods may be performed by any hardware apparatus.

While this invention has been described in terms of several embodiments,there are alterations, permutations, and equivalents which will beapparent to others skilled in the art and which fall within the scope ofthis invention. It should also be noted that there are many alternativeways of implementing the methods and compositions of the presentinvention. It is therefore intended that the following appended claimsbe interpreted as including all such alterations, permutations, andequivalents as fall within the true spirit and scope of the presentinvention.

What is claimed:
 1. A non-transitory computer-readable medium forstoring data associated with a video, comprising: a data stream storedin the non-transitory computer-readable medium, the data streamcomprising entropy-encoded information associated with a sample array ofa video, the entropy-encoded information including a plurality ofentropy slices, each of the plurality of entropy slices comprisingentropy-encoded data for a corresponding different portion of the samplearray, and the different portions forming rows of blocks of the samplearray, each row having a same number of blocks, wherein theentropy-encoded information is coded into the data stream for storing inthe non-transitory computer-readable medium by operations including:performing, for each entropy slice, entropy encoding along a respectiveentropy coding path using a respective probability estimation, whereinthe entropy coding path points in parallel along the rows of the blocks,adapting the respective probability estimation along the respectiveentropy coding path using a previously-encoded part of the respectiveentropy slice, and entropy encoding the plurality of entropy slicessequentially based on an entropy slice order, wherein entropy encoding apredetermined entropy slice includes entropy encoding a first block of afirst portion of the sample array corresponding to the predeterminedentropy slice along the entropy coding path by: initializing aprobability estimation of the first block based at least on aprobability estimation obtained after having entropy encoded a secondblock of a second portion of the sample array, wherein the secondportion is spatially-neighboring the first portion and corresponds to anentropy slice preceding, in the entropy slice order, the predeterminedentropy slice, and storing the probability estimation obtained afterhaving entropy encoded the second block.
 2. The non-transitorycomputer-readable medium according to claim 1, wherein the entropy sliceorder is chosen such that, along the entropy slice order, the differentportions of the sample array follow each other in a direction angledrelative to the entropy coding paths of the entropy slices.
 3. Thenon-transitory computer-readable medium according to claim 1, whereinthe stored probability estimation obtained after having entropy encodedthe second block is used for the initialization of the probabilityestimation of the first block.
 4. The non-transitory computer-readablemedium according to claim 1, the operations further includingperforming, for each entropy slice, entropy encoding along therespective entropy coding path and the adaptation of the respectiveprobability estimation along the respective entropy coding path suchthat, after a current part of the entropy slice has been entropy encodedbased on the respective probability estimation of the entropy slice, therespective probability estimation of the entropy slice is adapteddepending on the current part of the entropy slice, and a probabilityestimation obtained after entropy encoding a neighboring part of thepreceding entropy slice, the neighboring part spatially-neighboring thecurrent part.
 5. The non-transitory computer-readable medium accordingto claim 4, wherein the adaptation of the respective probabilityestimation of the entropy slice is performed by a first adaptationdepending on the current part of the entropy slice and an averaging of aresult of the first adaptation with the probability estimation obtainedafter entropy encoding the neighboring part.
 6. The non-transitorycomputer-readable medium according to claim 1, the operations furtherincluding steering the entropy encoding of immediately consecutiveentropy slices in the entropy slice order so that a distance ofcurrently encoded blocks of portions corresponding to the immediatelyconsecutive entropy slices measured in blocks along the encoding pathsis at least two blocks.
 7. The non-transitory computer-readable mediumaccording to claim 1, the operations further including steering theentropy encoding of immediately consecutive entropy slices in theentropy slice order so that a distance of currently encoded blocks ofportions corresponding to the immediately consecutive entropy slicesmeasured in blocks along the encoding paths is two blocks.
 8. Thenon-transitory computer-readable medium according to claim 1, theoperations further including sub-dividing the entropy slices intochunks, and interleaving the chunks, such that entropy decoding of theentropy slices is performed in parallel along entropy decoding pathseven before a reception of any of the entropy slices as a whole.
 9. Thenon-transitory computer-readable medium according to claim 1, whereinthe entropy slices are subdivided into chunks and the operations furtherinclude: checking whether a current chunk corresponds to a firstsub-portion of the portion of an entropy slice along the entropy codingpath, and if so, entropy encoding the current chunk by adapting therespective probability estimation and take a state of the respectiveprobability estimation, as obtained at an end of entropy encoding thecurrent chunk, into account when entropy encoding another chunk whichcorresponds to a second sub-portion of the portion of the entropy slicealong the entropy coding path, and if not, entropy encoding the currentchunk using a probability estimation which depend on a probabilityestimation obtained at an end of entropy encoding a chunk whichcorresponds to a sub-portion of the portion of the entropy slicepreceding the sub-portion corresponding to the current chunk, along theentropy coding path.
 10. The non-transitory computer-readable mediumaccording to claim 1, wherein the sample array is a current sample arrayof a sequence of sample arrays, the operations further including, inentropy encoding a predetermined entropy slice, entropy encoding acurrent part of the predetermined entropy slice based on the respectiveprobability estimation of the predetermined entropy slice as adaptedusing the previously encoded part of the predetermined entropy slice, aprobability estimation as used in the entropy encoding of a spatiallyneighboring, in the entropy slice order, preceding entropy slice at aneighboring part of the spatially neighboring preceding entropy slice,and a probability estimation used in encoding a previously encoded frameof the entropy encoded data stream relating to another sample array thanthe current sample array.